feat: CommonDevice → OpnSenseDocument reverse serializer#9
feat: CommonDevice → OpnSenseDocument reverse serializer#9unclesp1d3r merged 11 commits intomainfrom
Conversation
Pure-Go faker library used by the new internal/faker package to populate *model.CommonDevice with realistic hostnames, domains, and descriptions. Seeded via *rand.Rand for determinism. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Produces a fully-populated *model.CommonDevice from functional options (WithSeed, WithVLANCount, WithFirewallRules, WithHostname, WithDomain). Files mirror the CommonDevice subsystem layout: system.go, network.go (WAN/LAN + VLANs), dhcp.go, firewall.go. Seed 0 is the "non-deterministic" sentinel; any non-zero seed produces byte-stable output. No XML, no opnsense schema knowledge in this package — the CommonDevice model is the only output. Covers R1 (zero-arg path), R2 (CommonDevice is the intermediate representation), and R5 (deterministic output). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Implements the inverse of opnDossier's pkg/parser/opnsense.ConvertDocument. Serialize(*model.CommonDevice) returns a ready-to-marshal *opnsense.OpnSenseDocument with System, Interfaces, VLANs, Dhcpd, and Filter populated. Overlay layers generated content onto an existing base document, preserving fields Phase 1 does not own (NAT, VPN, OPNsense subsystem block, Theme, certificates, ...). One file per CommonDevice subsystem (system, interfaces, vlans, dhcp, firewall) so future subsystems plug in without restructuring. Package sits under internal/serializer/opnsense/ to reserve a sibling path for internal/serializer/pfsense/. Round-trip test exercises faker -> Serialize -> MarshalConfig -> ParseConfig -> ConvertDocument and asserts zero ConversionWarnings plus field-level parity. Covers R1, R2, R4 (overlay), R6 (package layout extensibility). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
internal/opnsensegen becomes transport-only (LoadBaseConfig, ParseConfig, MarshalConfig). The Inject* helpers are removed — generation lives in internal/faker, serialization in internal/serializer/opnsense. MarshalConfig post-processes the marshaled XML to sort children of map-backed sections (<interfaces>, <dhcpd>) alphabetically by tag name. Without this, opnDossier's map[string]T-backed MarshalXML emits children in Go's randomized map iteration order, breaking R5 (deterministic output under fixed seed). commondevice_test.go rewrites the consumer round-trip test to exercise the new pipeline. template_test.go drops tests for the deleted injection helpers. csvio.WriteVlanCSV now takes a *model.CommonDevice. IP range columns are derived by cross-referencing VLAN.VLANIf with Interface.PhysicalIf. Column order and German headers preserved. ReadVlanCSV dropped — unused after the CLI migration. Covers R2, R3 (CSV inspection path), R5 (determinism), R7 (no compatibility shims). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Default path: opnConfigGenerator generate with no arguments emits a valid OPNsense config.xml on stdout. Drops MarkFlagRequired on --format; xml is the default. New flag surface: --format xml (default) or csv --vlan-count/-n 0..4092 VLANs (default 10) --base-config optional overlay source --firewall-rules opt-in default allow-rules per interface --seed deterministic PCG seed (0 = random) --hostname override faker-generated hostname --domain override faker-generated domain --force, --output preserved from prior CLI Removed flags that served the deleted injection pipeline: --count (renamed to --vlan-count), --csv-file, --firewall-nr, --opt-counter, --include-firewall-rules, --firewall-rule-complexity, --vlan-range, --vpn-count, --nat-mappings, --wan-assignments. runGenerate composes faker.NewCommonDevice -> serializer.Serialize (or Overlay when --base-config is set) -> MarshalConfig. Covers R1 (zero-arg valid config.xml), R3 (CSV opt-in), R4 (--base-config overlay), R5 (deterministic seed). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
internal/generator (VlanConfig, DhcpServerConfig, FirewallRule, FirewallGenerator, DhcpGenerator, NatGenerator, VpnGenerator, departments) existed only to feed the deleted Inject* pipeline. internal/validate only validated generator.VlanConfig. Both are dead code under the CommonDevice pipeline. No consumers outside these directories remained after the CLI migration in the previous commits. Covers R7 (no compatibility shims for the old pipeline). Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Leads with the project's actual purpose: the missing inverse of opnDossier's parser. Zero-argument 'generate' is the headline example. New Pipeline section shows the mermaid flow. Command reference matches the new flag surface. "What Phase 1 Covers" table enumerates delivered subsystems alongside the deferred list so followup plans have a clear anchor. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
Autofixes driven by ce:review (mode:autofix) findings: Correctness (HIGH — silent round-trip field loss): * SerializeInterfaces now propagates iface.Type and iface.Virtual. Without these, every faker-generated opt VLAN interface lost Virtual=true and every interface lost its Type on the round-trip. Strengthened TestRoundTrip with per-field parity assertions that would have caught this; replaced count-only checks on Interfaces/VLANs/DHCP with lookup+equality. * endpointToSource/endpointToDestination no longer collapse empty Address onto <any/>. Empty means "unspecified" and now emits nothing; only Address == "any" emits the <any/> presence flag. Updated the relevant firewall test. Reliability: * MarshalConfig buffers the entire document before the first Write so a mid-stream encode or stabilization error no longer leaves a truncated file on disk. * pickUniqueTag and pickUniqueNet get a max-attempts cap with an explicit panic message on exhaustion. Today's pool sizes make this unreachable, but a future shrink in the address or tag space no longer hangs the CLI. * fakeDHCPScopes now logs a warning before silently skipping an interface whose IPAddress/Subnet cannot be parsed. Maintainability: * Dropped the unused *gofakeit.Faker parameter from fakeFirewallRules (YAGNI — the ruleset is deterministic today). * Lowercased NetworkResult → networkResult since nothing outside the faker package consumes it. Byte stability: * Added TestRoundTripByteStable — 10 re-marshals on the same input must produce identical bytes. Direct guard for the sortMapBackedSections workaround instead of relying on TestGenerateDeterministicSeed's two-run proxy. Docs: * GOTCHAS §7.1 captures the map-backed XML ordering workaround so future contributors adding NAT/VPN/etc. know where to extend mapBackedSections. * GOTCHAS §7.2 captures the round-trip-test-must-assert-per-field lesson. * CONTRIBUTING.md Project Structure and Key Design Decisions sections rewritten to describe the faker → CommonDevice → serializer/opnsense pipeline; removed references to the deleted internal/generator and internal/validate packages. * Added "Adding a New CommonDevice Subsystem" playbook so follow-up plans (NAT, VPN, Users, ...) have a concrete onboarding path. * Rewrote cmd/root.go Long description: removed VPN/NAT feature claims that were never implemented here and now clearly list Phase 1 scope. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
WalkthroughReworks generator into a faker→serializer→marshal pipeline: adds Changes
sequenceDiagram
participant User
participant CLI as cmd/generate
participant Faker as faker.NewCommonDevice
participant Device as model.CommonDevice
participant Serializer as serializer.Serialize
participant OPNDoc as opnsense.OpnSenseDocument
participant Marshal as opnsensegen.MarshalConfig
participant Stdout as stdout
User->>CLI: run generate (--vlan-count, --firewall-rules, --hostname)
CLI->>Faker: NewCommonDevice(opts...)
Faker->>Faker: fakeSystem / fakeNetwork / fakeDHCPScopes / fakeFirewallRules
Faker->>Device: return populated CommonDevice
CLI->>Serializer: Serialize(device)
Serializer->>OPNDoc: build OpnSenseDocument (System, Interfaces, VLANs, Dhcpd, Filter)
CLI->>Marshal: MarshalConfig(OPNDoc, w)
Marshal->>Marshal: encode buffer → sortMapBackedSections → stabilized bytes
Marshal->>Stdout: write header + stabilized body + newline
Stdout->>User: deterministic config.xml
Estimated code review effort🎯 4 (Complex) | ⏱️ ~75 minutes Possibly related PRs
Suggested labels
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
✨ Simplify code
Comment |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Pull request overview
This PR refactors opnConfigGenerator into a reverse-serializer pipeline that generates a synthetic *model.CommonDevice, serializes it into an *opnsense.OpnSenseDocument, and marshals it to a valid, byte-stable OPNsense config.xml (optionally overlaying onto a base config).
Changes:
- Added
internal/faker/to deterministically populate realistic*model.CommonDeviceinstances (seeded viagofakeit/v7+math/rand/v2). - Added
internal/serializer/opnsense/to serializeCommonDevice → OpnSenseDocument, including an overlay mode for--base-config. - Refactored
internal/opnsensegen/to transport-only responsibilities and added XML stabilization (sorting map-backed sections) for deterministic output; removed legacyinternal/generator/andinternal/validate/paths.
Reviewed changes
Copilot reviewed 55 out of 56 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| internal/validate/validate_test.go | Removes legacy validation tests tied to the deleted generator-based pipeline. |
| internal/validate/validate.go | Removes legacy cross-object validation logic tied to the deleted generator-based pipeline. |
| internal/serializer/opnsense/vlans_test.go | Adds unit tests for VLAN serialization into OPNsense schema. |
| internal/serializer/opnsense/vlans.go | Implements CommonDevice VLANs → opnsense.VLANs mapping. |
| internal/serializer/opnsense/system_test.go | Adds unit tests for system serialization defaults and field mapping. |
| internal/serializer/opnsense/system.go | Implements model.System → opnsense.System mapping with schema-required defaults. |
| internal/serializer/opnsense/serializer_test.go | Adds round-trip parity and byte-stability tests covering Phase 1 claims. |
| internal/serializer/opnsense/serializer.go | Adds Serialize(*CommonDevice) entry point with nil guarding. |
| internal/serializer/opnsense/overlay_test.go | Tests overlay behavior and nil error contracts. |
| internal/serializer/opnsense/overlay.go | Implements overlay that preserves base config outside Phase 1-owned fields. |
| internal/serializer/opnsense/interfaces_test.go | Adds tests ensuring interfaces serialize correctly and preserve round-trip fields. |
| internal/serializer/opnsense/interfaces.go | Implements interface slice → map-backed OPNsense interfaces serialization (preserving Type/Virtual). |
| internal/serializer/opnsense/firewall_test.go | Adds tests for filter rule serialization, especially “any” endpoint behavior. |
| internal/serializer/opnsense/firewall.go | Implements firewall rule serialization into opnsense.Filter. |
| internal/serializer/opnsense/dhcp_test.go | Adds tests for DHCP scope serialization keyed by interface. |
| internal/serializer/opnsense/dhcp.go | Implements DHCP scope → map-backed opnsense.Dhcpd serialization. |
| internal/opnsensegen/template_test.go | Updates tests to reflect transport-only opnsensegen and adds negative coverage (missing file, invalid XML). |
| internal/opnsensegen/template.go | Refactors to load/parse/marshal only and adds post-processing to sort map-backed XML sections for deterministic bytes. |
| internal/opnsensegen/commondevice_test.go | Updates end-to-end tests to validate the new reverse serializer pipeline via opnDossier round-trip. |
| internal/generator/vpn_test.go | Removes legacy VPN generator tests (generator package deleted). |
| internal/generator/vpn.go | Removes legacy VPN generator implementation. |
| internal/generator/vlan_test.go | Removes legacy VLAN generator tests. |
| internal/generator/vlan.go | Removes legacy VLAN generator implementation. |
| internal/generator/types.go | Removes legacy generator types. |
| internal/generator/nat_test.go | Removes legacy NAT generator tests. |
| internal/generator/nat.go | Removes legacy NAT generator implementation. |
| internal/generator/helpers_test.go | Removes generator test helper. |
| internal/generator/firewall_test.go | Removes legacy firewall generator tests. |
| internal/generator/firewall.go | Removes legacy firewall generator implementation. |
| internal/generator/dhcp_test.go | Removes legacy DHCP derivation tests. |
| internal/generator/dhcp.go | Removes legacy DHCP derivation implementation. |
| internal/generator/departments_test.go | Removes legacy department generator tests. |
| internal/generator/departments.go | Removes legacy department generator implementation. |
| internal/faker/system_test.go | Adds determinism and schema-validity tests for fake system generation. |
| internal/faker/system.go | Implements schema-valid system field generation (hostname/domain/timezone/DNS/NTP). |
| internal/faker/rand_test.go | Adds tests for deterministic vs non-deterministic seeding behavior. |
| internal/faker/rand.go | Implements shared seeded RNG + gofakeit faker stream creation. |
| internal/faker/options.go | Adds functional options controlling seed, VLAN count, firewall rules, and hostname/domain overrides. |
| internal/faker/network_test.go | Adds tests for WAN/LAN/VLAN interface + VLAN generation behavior and determinism. |
| internal/faker/network.go | Implements network shaping (WAN DHCP, LAN static RFC1918, VLAN-backed opt interfaces). |
| internal/faker/firewall_test.go | Adds tests for minimal “default allow” per-interface firewall rules generation. |
| internal/faker/firewall.go | Implements minimal default firewall rule emission for non-WAN interfaces. |
| internal/faker/dhcp_test.go | Adds tests ensuring one DHCP scope per static interface and correct skipping behavior. |
| internal/faker/dhcp.go | Implements DHCP scope generation from interface addressing. |
| internal/faker/device_test.go | Adds tests for NewCommonDevice defaults, determinism, VLAN counts, and overrides. |
| internal/faker/device.go | Implements NewCommonDevice pipeline stitching system/network/dhcp/firewall into CommonDevice. |
| internal/csvio/csvio_test.go | Updates CSV tests to write VLAN CSVs from *model.CommonDevice rather than legacy generator types. |
| internal/csvio/csvio.go | Refactors CSV output to be derived from CommonDevice (preserving German headers + BOM). |
| go.sum | Adds checksums for the new faker dependency. |
| go.mod | Adds github.com/brianvoe/gofakeit/v7 dependency. |
| cmd/root.go | Updates CLI help text to reflect reverse-serializer purpose and overlay behavior. |
| cmd/generate.go | Reshapes CLI flags/defaults; makes zero-arg XML generation work; routes through faker/serializer and optional overlay. |
| cmd/cmd_test.go | Updates command tests for new CLI behavior (zero-arg XML, overlay, determinism, new flags). |
| README.md | Reframes project documentation around CommonDevice reverse serialization and new CLI usage. |
| GOTCHAS.md | Documents determinism pitfalls with map-backed XML sections and required round-trip parity testing. |
| CONTRIBUTING.md | Updates architecture documentation to reflect faker + serializer + transport layering. |
Comments suppressed due to low confidence (1)
cmd/generate.go:98
getOutputWriter()is called before XML serialization and before loading--base-config. When--outputis a file, this creates/truncates the destination even if a later step fails (e.g., base config read/parse error, overlay/serialize error), leaving an empty/partial file behind. Consider deferring file creation until after the document is fully prepared (or marshal into memory / a temp file and atomically rename) so failures don’t produce orphaned output files.
w, needClose, err := getOutputWriter()
if err != nil {
return err
}
if needClose {
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 14
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
cmd/generate.go (1)
94-104:⚠️ Potential issue | 🟠 MajorOpen the destination only after rendering succeeds.
The output writer is acquired before base-config loading, overlaying, and XML/CSV rendering. If any of those steps fail, an
--outputtarget can already be created or truncated even though no valid artifact was produced. Render into memory first, then open/write the final destination once the bytes are ready.Also applies to: 118-129
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cmd/generate.go` around lines 94 - 104, The current code opens the output destination via getOutputWriter() before performing base-config loading, overlaying, and XML/CSV rendering, which can create/truncate the target even if rendering fails; change the flow so rendering happens into memory first (e.g., produce a bytes.Buffer or []byte from the base-config loading, overlaying, and XML/CSV rendering steps), and only after rendering succeeds call getOutputWriter(), write the rendered bytes, and then close the writer (handling close errors as before). Apply the same change to the other opening/writing block mentioned (the similar code at the 118-129 region) so no output file is created/truncated before successful rendering.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@cmd/generate.go`:
- Around line 75-84: The CLI currently allows using --base-config with non-XML
output and silently ignores it; update the validation in the generate command
(after computing format via normalizeStringFlag(outputFormat) and before
returning success) to reject any invocation that supplies baseConfig when format
is not formatXML (i.e., when format == formatCSV or default), returning a clear
error such as "--base-config is only valid with xml format"; check the flag
variable (baseConfig or baseConfigPath) and return fmt.Errorf when it's set and
format != formatXML so the CLI fails fast and informs the user.
In `@CONTRIBUTING.md`:
- Around line 120-123: The documentation currently claims a dispatcher exists
but the CLI still hardwires the OPNsense serializer in cmd/generate.go; update
the text to describe the intended architecture as planned behavior: state that
model.CommonDevice is intended as the single IR, that internal/faker/ will
populate it and internal/serializer/opnsense/ implements one serializer, and
that the package layout reserves a pfSense sibling so a future
internal/serializer/pfsense/ can plug in and the CLI will route by
device.DeviceType once implemented; do not assert that the dispatcher or device
routing is already present.
In `@internal/csvio/csvio.go`:
- Around line 46-49: The current map byPhysical :=
make(map[string]model.Interface...) uses iface.PhysicalIf as key which causes
collisions when PhysicalIf == ""; change byPhysical to
map[string][]model.Interface and populate with byPhysical[iface.PhysicalIf] =
append(byPhysical[iface.PhysicalIf], iface) so multiple interfaces (including
empty keys) are preserved; then update any downstream code that reads byPhysical
to handle a slice of model.Interface instead of a single model.Interface.
In `@internal/faker/device_test.go`:
- Around line 34-41: Add edge-case tests for VLAN count boundaries by creating
new tests that call NewCommonDevice with WithVLANCount(0) and WithVLANCount near
the upper limit (e.g. 4092); for the zero case assert dev.VLANs is empty,
dev.Interfaces length equals 2 (WAN + LAN) and dev.DHCP length equals 1, and for
the high-count case assert dev.VLANs length matches the requested count,
dev.Interfaces grows by that count (2 + count), and VLAN tag uniqueness (no
duplicates) across dev.VLANs; reference TestNewCommonDeviceVLANCount,
NewCommonDevice, WithVLANCount, dev.VLANs, dev.Interfaces, and dev.DHCP when
adding these tests.
In `@internal/faker/network.go`:
- Around line 35-39: The fakeNetwork function currently assumes vlanCount is
valid and downstream helpers pickUniqueTag/pickUniqueNet may panic on exhausted
or out-of-range VLAN space; validate vlanCount in fakeNetwork (and/or at the
start of NewCommonDevice) to ensure 0 <= vlanCount <= 4092 and return a
descriptive error instead of proceeding, change the fakeNetwork signature to
return (networkResult, error) and propagate errors from
pickUniqueTag/pickUniqueNet back to the caller (adjust NewCommonDevice to handle
the error), and update any callers of fakeNetwork to handle the error path so
invalid counts are rejected with clear messages rather than panicking.
In `@internal/faker/options.go`:
- Around line 1-38: NewCommonDevice must validate cfg.vlanCount before calling
fakeNetwork: reject values <0 with error "VLAN count must be >= 0" and >4092
with error "VLAN count must be <= 4092". Change NewCommonDevice to return
(*model.CommonDevice, error) (update its callers) and perform the checks on
cfg.vlanCount early; if invalid, return nil and the corresponding error,
otherwise proceed to call fakeNetwork and return the created *model.CommonDevice
and nil error. Ensure the validation references cfg.vlanCount and the
NewCommonDevice function name so future edits are clear.
In `@internal/faker/rand_test.go`:
- Around line 25-31: The test TestNewRandZeroSeedIsRandom is flaky because it
compares a single Uint64() draw from two streams; change it to sample multiple
times (e.g., loop N times) from the two rand.Rand instances returned by
newRand(0) and assert that at least one pair of corresponding draws differs (or
assert divergence within a small number of draws), referencing newRand and
TestNewRandZeroSeedIsRandom so the test is hardened against probabilistic
collisions while preserving the intent that zero-seed streams diverge.
In `@internal/faker/system_test.go`:
- Around line 49-59: TestFakeSystemDomainIsLowercaseFQDN currently only uses
seed 17; update it to iterate the same seed set used in
TestFakeSystemHostnameIsDNSSafe (e.g., {1,2,3,42,100}) and run the newRand(seed)
-> fakeSystem(f) flow for each seed so sys.Domain is validated across multiple
seeds; keep the same assertions (no uppercase runes and contains a dot) and
reference the existing helpers newRand and fakeSystem while iterating seeds
inside TestFakeSystemDomainIsLowercaseFQDN.
In `@internal/opnsensegen/commondevice_test.go`:
- Around line 49-56: Add a few extra assertions in commondevice_test.go after
the existing checks to spot-check additional fields for stronger round-trip
parity: assert that device.System.Timezone and device.System.Language equal
original.System.Timezone and original.System.Language (use assert.Equal with the
same pattern as Hostname/Domain), and inspect VLANs on device.VLANs to assert
the expected tags/IDs for at least one or two entries (e.g., check
device.VLANs[0].Tag and .ID or a VLAN with a known name), and optionally assert
a specific interface property from device.Interfaces (like a first interface
Name or MAC) to ensure deeper coverage while keeping the zero-warnings gate.
In `@internal/serializer/opnsense/dhcp_test.go`:
- Around line 35-40: The test TestSerializeDHCPDisabled is currently asserting
out.Items["lan"].Enable which will succeed if the "lan" key is missing due to
Go's zero-value behavior; change the test to first verify the key exists in the
map returned by SerializeDHCP (e.g., check that "lan" is present using a map
lookup or assert.Contains) and only then assert that out.Items["lan"].Enable
indicates disabled; reference SerializeDHCP, TestSerializeDHCPDisabled and the
out.Items["lan"].Enable lookup when making the change.
In `@internal/serializer/opnsense/interfaces_test.go`:
- Around line 51-56: The test TestSerializeInterfacesDisabled uses
serializer.SerializeInterfaces and then directly asserts fields on
out.Items["lan"], which can falsely pass if the "lan" key is missing; update the
test to first assert that the map contains the expected key (e.g., check
presence of "lan" in out.Items using a key-existence assertion) and only then
assert the Enable field value; apply the same change to the other negative-path
test(s) that reference "opt1" so they verify key presence before inspecting
fields.
In `@internal/serializer/opnsense/overlay.go`:
- Around line 23-25: The Serialize(device) call in overlay.go returns err which
is propagated directly; change the error return to wrap it with context (e.g.,
use fmt.Errorf) so failures show overlay serialization context — update the
return path after serialized, err := Serialize(device) to return nil,
fmt.Errorf("overlay serialize: %w", err) (and add the fmt import if missing).
In `@internal/serializer/opnsense/vlans_test.go`:
- Around line 11-26: Test covers first VLAN only; add assertions for the second
serialized VLAN to validate iteration: after calling
serializer.SerializeVLANs(in) in TestSerializeVLANs, add at least a spot-check
like assert.Equal(t, "100", out.VLAN[1].Tag) (optionally also assert.If/Descr
using out.VLAN[1].If and out.VLAN[1].Descr) to confirm the second entry was
serialized correctly.
In `@README.md`:
- Line 52: Update the README sentence to clarify that routing by
CommonDevice.DeviceType is planned rather than implemented: change the
description around the generate CLI path to state that currently the CLI still
hardwires the opnsense serializer (internal/serializer/opnsense) but will in
future dispatch to a serializer based on CommonDevice.DeviceType, allowing a
sibling internal/serializer/pfsense to be selected; reference the generate
command/CLI routing and the CommonDevice.DeviceType selector so readers know
this is intended behavior, not current functionality.
---
Outside diff comments:
In `@cmd/generate.go`:
- Around line 94-104: The current code opens the output destination via
getOutputWriter() before performing base-config loading, overlaying, and XML/CSV
rendering, which can create/truncate the target even if rendering fails; change
the flow so rendering happens into memory first (e.g., produce a bytes.Buffer or
[]byte from the base-config loading, overlaying, and XML/CSV rendering steps),
and only after rendering succeeds call getOutputWriter(), write the rendered
bytes, and then close the writer (handling close errors as before). Apply the
same change to the other opening/writing block mentioned (the similar code at
the 118-129 region) so no output file is created/truncated before successful
rendering.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 911f1b78-e712-4569-8fe8-2d9ab759ed41
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (55)
CONTRIBUTING.mdGOTCHAS.mdREADME.mdcmd/cmd_test.gocmd/generate.gocmd/root.gogo.modinternal/csvio/csvio.gointernal/csvio/csvio_test.gointernal/faker/device.gointernal/faker/device_test.gointernal/faker/dhcp.gointernal/faker/dhcp_test.gointernal/faker/firewall.gointernal/faker/firewall_test.gointernal/faker/network.gointernal/faker/network_test.gointernal/faker/options.gointernal/faker/rand.gointernal/faker/rand_test.gointernal/faker/system.gointernal/faker/system_test.gointernal/generator/departments.gointernal/generator/departments_test.gointernal/generator/dhcp.gointernal/generator/dhcp_test.gointernal/generator/firewall.gointernal/generator/firewall_test.gointernal/generator/helpers_test.gointernal/generator/nat.gointernal/generator/nat_test.gointernal/generator/types.gointernal/generator/vlan.gointernal/generator/vlan_test.gointernal/generator/vpn.gointernal/generator/vpn_test.gointernal/opnsensegen/commondevice_test.gointernal/opnsensegen/template.gointernal/opnsensegen/template_test.gointernal/serializer/opnsense/dhcp.gointernal/serializer/opnsense/dhcp_test.gointernal/serializer/opnsense/firewall.gointernal/serializer/opnsense/firewall_test.gointernal/serializer/opnsense/interfaces.gointernal/serializer/opnsense/interfaces_test.gointernal/serializer/opnsense/overlay.gointernal/serializer/opnsense/overlay_test.gointernal/serializer/opnsense/serializer.gointernal/serializer/opnsense/serializer_test.gointernal/serializer/opnsense/system.gointernal/serializer/opnsense/system_test.gointernal/serializer/opnsense/vlans.gointernal/serializer/opnsense/vlans_test.gointernal/validate/validate.gointernal/validate/validate_test.go
💤 Files with no reviewable changes (16)
- internal/generator/helpers_test.go
- internal/generator/dhcp_test.go
- internal/validate/validate_test.go
- internal/generator/nat_test.go
- internal/generator/vlan_test.go
- internal/generator/vpn_test.go
- internal/generator/firewall_test.go
- internal/validate/validate.go
- internal/generator/vpn.go
- internal/generator/departments.go
- internal/generator/nat.go
- internal/generator/vlan.go
- internal/generator/types.go
- internal/generator/dhcp.go
- internal/generator/departments_test.go
- internal/generator/firewall.go
Addresses PR #9 secondary review findings beyond the initial ce:review pass. Reliability: * pickUniqueTag and pickUniqueNet now return (value, error) instead of panicking on exhaustion. fakeNetwork, fakeDHCPScopes, and NewCommonDevice propagate the error. cmd/generate wraps it as "generate device: %w" so a pathological seed + vlan-count surfaces as a clean CLI error instead of a Go stack trace. * fakeDHCPScopes upgrades the ParsePrefix failure from log.Warn (silently suppressed by --quiet) to a returned error. An unparseable IP/subnet in the faker's input is a bug, not a recoverable condition. * emitSortedChildren now warns when it drops non-whitespace tokens (comments, CDATA, processing instructions, directives) between children of a map-backed section. Operators authoring an annotated base config see the drop rather than discovering it on a diff. Tests: * TestRoundTrip now asserts per-field parity on FirewallRules (Type/Description/IPProtocol/Source.Address/Destination.Address) to match the parity checks on Interface/VLAN/DHCP. * TestOverlayPreservesBaseConfigUnrelatedFields now populates Sysctl and CAs in the base and asserts they survive the overlay. The old empty-base fixture couldn't detect a field-preservation regression. * TestMarshalConfigIsAtomicOnSuccess verifies exactly one Write call on success; TestMarshalConfigDoesNotWriteOnWriterFailure pins zero partial output on failure. Both lock in the "atomic or nothing" contract that MarshalConfig's docstring promises. * TestMarshalConfigSortsMapBackedSections feeds interfaces named zeta/alpha/mu and asserts alphabetical order in the output — a direct unit test for the sort post-processor instead of proxying through the faker. * TestMarshalConfigHandlesEmptyMapBackedSections covers empty interfaces and dhcpd sections. * TestMarshalConfigByteStableMapIteration runs 20 re-marshals (up from 10 in the previous serializer test) as a stronger defeat-randomization guard. * TestPickUniqueTagReturnsErrorOnExhaustion pre-populates the used set with every valid tag and asserts the error path. * TestFakeDHCPScopesErrorOnUnparseablePrefix pins the new error-returning contract. * TestNewCommonDeviceFuzzSeeds loops 200 distinct seeds with 8 VLANs to catch regressions under adversarial seed streams. * cmd/cmd_test.go adds boundary tests: --vlan-count 0, 1, 4093 (reject), malformed --base-config XML, and a pin on Overlay's wholesale-replace semantic for <filter>. Docs / comments: * cmd/validate.go Long no longer advertises features the subcommand doesn't have. It now states "not yet implemented — reserved for a future phase". * internal/faker/rand.go drops the duplicated seed-zero prose and points to WithSeed as the single source of truth. * internal/faker/firewall.go clarifies that a bare interface name in RuleEndpoint.Address relies on OPNsense's network-alias resolution. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 4
♻️ Duplicate comments (2)
cmd/generate.go (1)
74-84:⚠️ Potential issue | 🟡 MinorReject
--base-configwhen--formatis notxml.
generate --format csv --base-config ...currently succeeds, but Line 121 is never reached in CSV mode, so the base config is silently ignored. Fail fast in validation so the CLI does not accept a no-op flag combination.♻️ Proposed fix
func runGenerate(_ *cobra.Command, _ []string) (err error) { format := normalizeStringFlag(outputFormat) switch format { case formatXML, formatCSV: default: return fmt.Errorf("invalid format %q: must be xml or csv", outputFormat) } if vlanCount < 0 || vlanCount > maxVlanCount { return fmt.Errorf("--vlan-count must be between 0 and %d, got %d", maxVlanCount, vlanCount) } + if format != formatXML && baseConfigPath != "" { + return fmt.Errorf("--base-config is only supported with --format %s", formatXML) + }As per coding guidelines "CLI command files: Focus on user experience, clear error messages, and proper flag handling."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@cmd/generate.go` around lines 74 - 84, The CLI currently allows --base-config with non-xml output because validation for base-config is never performed in runGenerate; add an explicit check in runGenerate (after determining format from normalizeStringFlag(outputFormat) and the format switch) to return an error when the user passed the base-config flag (e.g., variable representing --base-config) while format != formatXML; use a clear message like "`--base-config` is only valid with --format xml`" and return fmt.Errorf(...) so the command fails fast instead of silently ignoring the flag.internal/faker/network.go (1)
39-43:⚠️ Potential issue | 🔴 CriticalValidate
vlanCountbefore the slice allocations.Line 41 can panic on negative input before your error-return path ever runs, and direct callers of
NewCommonDevicecan also bypass the CLI’s0..4092guard and spend time in the exhaustion loops. Reject the range up front here so the faker API fails normally instead of crashing or retrying needlessly.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/faker/network.go` around lines 39 - 43, The function fakeNetwork currently allocates slices using vlanCount before validating it, which can panic for negative values or allow callers (including NewCommonDevice) to bypass CLI guards and cause wasted retries; add an upfront validation in fakeNetwork that checks vlanCount is within the allowed range (e.g., 0..4092) and return an error immediately if out of range, before constructing networkResult and allocating Interfaces/VLANs, so callers receive a proper error instead of a panic or unnecessary loops.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/faker/dhcp_test.go`:
- Around line 21-38: The test currently only checks DHCP range endpoints; update
the happy-path in dhcp_test.go to also assert each scope's Gateway and DNSServer
are set and correct: for each scope returned by fakeDHCPScopes, assert s.Gateway
and s.DNSServer are non-empty, parseable via netip.ParseAddr, and match the
interface IP that was used to create that scope (use the original interfaces
input to look up the expected IP for the scope.Interface name). Keep these
checks inside the existing loop that validates s.Range so they run per-scope.
In `@internal/opnsensegen/commondevice_test.go`:
- Around line 31-57: The test never exercises filter/firewall rules because
faker.NewCommonDevice was called without enabling rules; update the
faker.NewCommonDevice call to include faker.WithFirewallRules(true) alongside
faker.WithSeed(146) and faker.WithVLANCount(2), then after ConvertDocument
succeeds add an assertion that at least one rule survived (e.g.
require.NotEmpty(t, device.Firewall.Rules) or assert.NotEmpty(t,
device.Firewall.Rules)) to guarantee the serializer → opnsensegen.MarshalConfig
→ opnsensegen.ParseConfig → opnsenseparser.ConvertDocument round-trip preserves
firewall rules.
In `@internal/serializer/opnsense/overlay_test.go`:
- Around line 16-44: The test only verifies System and VLANs but not
Dhcpd/Filter so seed both Dhcpd and Filter in the base OpnSenseDocument fixture
(the base variable of type opnschema.OpnSenseDocument) with representative
items, recreate the device via faker.NewCommonDevice using
faker.WithFirewallRules(true) (in addition to faker.WithVLANCount(2)) so the
device contains firewall/DHCP state, call serializer.Overlay(base, device) as
before, and add assertions that merged.Dhcpd (or merged.Dhcpd.* fields you
seeded) and merged.Filter (or merged.Filter.* rules) equal the corresponding
values from device to ensure Overlay replaces those Phase 1 subsystems from the
device.
In `@internal/serializer/opnsense/serializer_test.go`:
- Around line 92-105: The current per-field parity loop in TestRoundTrip (using
firewallRulesByInterface to map original.FirewallRules and
roundTripped.FirewallRules) only checks Type/Description/IPProtocol/Addresses;
add assert.Equalf checks inside the same for iface, want := range origRules loop
to compare want.Direction vs got.Direction, want.Protocol vs got.Protocol,
want.Log vs got.Log, want.Disabled vs got.Disabled, want.Tracker vs got.Tracker
and also compare want.Source.Port vs got.Source.Port, want.Source.Negated vs
got.Source.Negated, want.Destination.Port vs got.Destination.Port and
want.Destination.Negated vs got.Destination.Negated (use the same assert.Equalf
pattern and message format as the existing assertions so missing fields in
SerializeFilter are detected).
---
Duplicate comments:
In `@cmd/generate.go`:
- Around line 74-84: The CLI currently allows --base-config with non-xml output
because validation for base-config is never performed in runGenerate; add an
explicit check in runGenerate (after determining format from
normalizeStringFlag(outputFormat) and the format switch) to return an error when
the user passed the base-config flag (e.g., variable representing --base-config)
while format != formatXML; use a clear message like "`--base-config` is only
valid with --format xml`" and return fmt.Errorf(...) so the command fails fast
instead of silently ignoring the flag.
In `@internal/faker/network.go`:
- Around line 39-43: The function fakeNetwork currently allocates slices using
vlanCount before validating it, which can panic for negative values or allow
callers (including NewCommonDevice) to bypass CLI guards and cause wasted
retries; add an upfront validation in fakeNetwork that checks vlanCount is
within the allowed range (e.g., 0..4092) and return an error immediately if out
of range, before constructing networkResult and allocating Interfaces/VLANs, so
callers receive a proper error instead of a panic or unnecessary loops.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: b3d42254-5c73-4a4a-9a26-9f6b2b1dab95
📒 Files selected for processing (17)
cmd/cmd_test.gocmd/generate.gocmd/validate.gointernal/csvio/csvio_test.gointernal/faker/device.gointernal/faker/device_test.gointernal/faker/dhcp.gointernal/faker/dhcp_test.gointernal/faker/firewall.gointernal/faker/network.gointernal/faker/network_test.gointernal/faker/rand.gointernal/opnsensegen/commondevice_test.gointernal/opnsensegen/template.gointernal/opnsensegen/template_test.gointernal/serializer/opnsense/overlay_test.gointernal/serializer/opnsense/serializer_test.go
Eighteen unresolved review threads from coderabbitai and the Copilot reviewer — fixes below cover the full set. CLI validation: * maxVlanCount was off-by-one vs the faker's tag pool. The 802.1Q range [2..4094] holds 4093 unique tags; bump maxVlanCount to 4093 and update the --vlan-count flag help + README + cmd_test boundary case. The constant now aliases faker.MaxVLANCount so the CLI and library share one source of truth. * Reject --base-config when --format is not xml. The overlay path only runs for XML; silently ignoring the flag for csv is a usability trap. Tests pin the rejection. Faker library hardening: * NewCommonDevice now validates vlan count (>=0, <=MaxVLANCount) before calling fakeNetwork, with explicit error messages. Defense in depth — CLI already bounds at the flag boundary, but the library is directly reachable by future consumers. Overlay error wrapping: * Wrap the Serialize error with "overlay: serialize device: %w" so callers see operation context in logs. CSV robustness: * Skip interfaces with empty PhysicalIf when indexing (they can never match a VLAN.VLANIf anyway). Extracted indexByPhysical and ipRangeFor helpers to keep the package average complexity under the linter bound. Docs drift (dispatcher described as current, not planned): * README and CONTRIBUTING reworded to say the CLI hardwires the OPNsense serializer today; CommonDevice.DeviceType-based routing lands when the pfSense sibling ships. Test hardening — mostly key-presence and per-field parity gaps: * TestNewRandZeroSeedIsRandom samples 8 draws instead of 1 (flake probability drops from ~2^-64 to ~2^-512). * TestFakeSystemDomainIsLowercaseFQDN runs across 5 seeds, matching TestFakeSystemHostnameIsDNSSafe's coverage. * TestFakeDHCPScopesOnePerStaticInterface pins Gateway and DNSServer (previously only range endpoints were checked). * NewCommonDevice gains boundary tests: VLANCount=0, negative (reject), > MaxVLANCount (reject). * Serializer dhcp/interfaces tests require.True on map key presence before asserting fields — a regression that drops the entry entirely would now fail loud. * TestSerializeVLANs spot-checks the second entry (previously only first-entry fields were validated). * TestCommonDeviceRoundTripViaSerializer now enables WithFirewallRules, asserts the rule count survives round-trip, and spot-checks Timezone + Language + per-VLAN tag identity. * New TestOverlayReplacesDhcpdAndFilter: seeds base Dhcpd + Filter with distinct sentinel values, enables faker firewall rules, asserts base values are dropped wholesale. * TestRoundTrip firewall parity now includes Direction, Protocol, Log, Disabled, Tracker, Source/Destination.Port, Source/Destination.Negated. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 56 out of 57 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 4
♻️ Duplicate comments (2)
internal/serializer/opnsense/serializer_test.go (1)
97-113:⚠️ Potential issue | 🟠 MajorInclude
FirewallRule.Interfacesin the parity loop.Keying by
Interfaces[0]only proves the first interface survived. A round-trip that widens, reorders, or otherwise mutates the interface list still passes here. Add a direct equality check inside this loop.As per coding guidelines, `**/*_test.go`: `Verify tests cover critical functionality and real edge cases`.♻️ Suggested parity addition
for iface, want := range origRules { got, ok := rtRules[iface] require.Truef(t, ok, "firewall rule for interface %q missing on round-trip", iface) + assert.Equalf(t, want.Interfaces, got.Interfaces, "rule %q Interfaces", iface) assert.Equalf(t, want.Type, got.Type, "rule %q Type", iface) assert.Equalf(t, want.Description, got.Description, "rule %q Description", iface)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/serializer/opnsense/serializer_test.go` around lines 97 - 113, The parity loop currently compares rule fields keyed by iface but misses verifying the full Interfaces slice; update the loop that iterates over origRules (for iface, want := range origRules) to also assert equality of want.Interfaces and got.Interfaces (the FirewallRule.Interfaces field) for each rule, ensuring order and contents match exactly between want and got; use the existing test helpers (e.g., assert.Equalf or require.Equalf) referencing the variables want.Interfaces and got.Interfaces and include a descriptive message like "rule %q Interfaces" to make failures clear.internal/serializer/opnsense/overlay_test.go (1)
86-98:⚠️ Potential issue | 🟠 MajorAssert overlay output equals the device payload, not just
!=base.These checks only prove the base values disappeared. The test still stays green if
Overlaywrites the wrongDhcpdorFilter, as long as it differs from the fixture. Compare the merged sections to the serialized device output so wholesale replacement is actually gated.As per coding guidelines, `**/*_test.go`: `Verify tests cover critical functionality and real edge cases`.♻️ Stronger wholesale-replacement assertion
merged, err := serializer.Overlay(base, device) require.NoError(t, err) + serialized, err := serializer.Serialize(device) + require.NoError(t, err) - // Dhcpd — base's lan gateway 10.99.99.1 must NOT survive; the device's - // LAN gateway replaces it. - lan, ok := merged.Dhcpd.Items["lan"] - require.True(t, ok, "merged document must have a lan dhcpd entry from the device") - assert.NotEqual(t, "10.99.99.1", lan.Gateway, - "base Dhcpd must be replaced wholesale; base gateway must not survive") + assert.Equal(t, serialized.Dhcpd, merged.Dhcpd, "Dhcpd must be replaced wholesale from device") - // Filter — the base's block rule must not survive. - for _, r := range merged.Filter.Rule { - assert.NotEqual(t, "from base — must be dropped", r.Descr, - "base Filter.Rule entries must be replaced wholesale") - } - assert.NotEmpty(t, merged.Filter.Rule, "device firewall rules must land in merged Filter") + assert.Equal(t, serialized.Filter, merged.Filter, "Filter must be replaced wholesale from device")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@internal/serializer/opnsense/overlay_test.go` around lines 86 - 98, The test currently only asserts base values were removed (using merged.Dhcpd.Items["lan"] and merged.Filter.Rule) but must assert the merged sections equal the device payload; update the assertions to compare merged.Dhcpd.Items["lan"] (including Gateway) to the device's Dhcpd "lan" entry from the serialized device fixture and assert merged.Filter.Rule equals (or deep-equals) the device's Filter.Rule slice, replacing the NotEqual checks so Overlay wholesale-replacement is validated against the actual device document rather than merely being different from the base.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@cmd/cmd_test.go`:
- Around line 22-35: The test helper fails to reset three package-level validate
flags—inputFile, inputFormat, and maxErrors—so state leaks between tests; update
the reset block in newTestRootCmd (or wherever the other flag globals like
outputFormat, vlanCount, includeFirewall, etc. are reset) to also assign
inputFile = "", inputFormat = <defaultInputFormat> (or the correct default
constant), and maxErrors = <defaultMaxErrors> so validate-related tests are
order-independent.
In `@cmd/generate.go`:
- Around line 123-132: The current code always calls
serializer.Serialize(device) and then discards its result when baseConfigPath is
provided; instead, branch on baseConfigPath before serializing so the device is
only serialized once: if baseConfigPath != "" first call
opnsensegen.LoadBaseConfig(baseConfigPath) and then call
serializer.Overlay(base, device) to produce doc; otherwise call
serializer.Serialize(device) to produce doc. Update the logic around
baseConfigPath, serializer.Serialize, serializer.Overlay and
opnsensegen.LoadBaseConfig so doc is only produced once from the appropriate
path.
In `@internal/faker/device.go`:
- Around line 59-66: The faker currently hard-codes model.DeviceTypeOPNsense
into the intermediate CommonDevice struct in NewCommonDevice; remove that
target-specific assignment so NewCommonDevice returns a device-agnostic
*model.CommonDevice with DeviceType left unset (zero value) instead of
model.DeviceTypeOPNsense, and ensure downstream serializers are responsible for
setting DeviceType; update any callers/tests that incorrectly assume
NewCommonDevice populated DeviceType to use serializer injection or explicit
setting before serialization.
In `@README.md`:
- Around line 24-26: The fenced code block showing the shell command
"opnconfiggenerator generate --base-config existing.xml --seed 42" in README.md
is missing a language specifier; update that opening triple-backtick to include
a language (use "text" or "console") so static analysis passes (e.g., change ```
to ```text) and apply the same fix to the similar block around line 66.
---
Duplicate comments:
In `@internal/serializer/opnsense/overlay_test.go`:
- Around line 86-98: The test currently only asserts base values were removed
(using merged.Dhcpd.Items["lan"] and merged.Filter.Rule) but must assert the
merged sections equal the device payload; update the assertions to compare
merged.Dhcpd.Items["lan"] (including Gateway) to the device's Dhcpd "lan" entry
from the serialized device fixture and assert merged.Filter.Rule equals (or
deep-equals) the device's Filter.Rule slice, replacing the NotEqual checks so
Overlay wholesale-replacement is validated against the actual device document
rather than merely being different from the base.
In `@internal/serializer/opnsense/serializer_test.go`:
- Around line 97-113: The parity loop currently compares rule fields keyed by
iface but misses verifying the full Interfaces slice; update the loop that
iterates over origRules (for iface, want := range origRules) to also assert
equality of want.Interfaces and got.Interfaces (the FirewallRule.Interfaces
field) for each rule, ensuring order and contents match exactly between want and
got; use the existing test helpers (e.g., assert.Equalf or require.Equalf)
referencing the variables want.Interfaces and got.Interfaces and include a
descriptive message like "rule %q Interfaces" to make failures clear.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: af3f6064-9c34-4aa7-af39-1147248239ed
📒 Files selected for processing (17)
CONTRIBUTING.mdREADME.mdcmd/cmd_test.gocmd/generate.gointernal/csvio/csvio.gointernal/faker/device.gointernal/faker/device_test.gointernal/faker/dhcp_test.gointernal/faker/rand_test.gointernal/faker/system_test.gointernal/opnsensegen/commondevice_test.gointernal/serializer/opnsense/dhcp_test.gointernal/serializer/opnsense/interfaces_test.gointernal/serializer/opnsense/overlay.gointernal/serializer/opnsense/overlay_test.gointernal/serializer/opnsense/serializer_test.gointernal/serializer/opnsense/vlans_test.go
Round 2 of automated reviewer feedback. CLI cleanup: * Factor out buildOpnSenseDocument so the XML path branches before serializing. With --base-config, only Overlay runs (which serializes internally); without it, only Serialize runs. The old path ran Serialize unconditionally and then discarded the result on the overlay branch. * Reset validate subcommand globals (inputFile, inputFormat, maxErrors) in newTestRootCmd alongside the other CLI globals so validate tests do not leak state across invocations. Faker architecture: * NewCommonDevice no longer hard-codes model.DeviceTypeOPNsense on the produced device. Added WithDeviceType option; the CLI sets OPNsense explicitly. Keeps the faker target-neutral so future pfSense tests (and any other consumer) can choose their own type. Round-trip tests updated to pass WithDeviceType(OPNsense) where they assert on the type. Docstring accuracy: * template.go emitSortedChildren: the code logs-and-drops non-whitespace tokens between children. The comment claimed "preserved-but-warned" which was a lie. Rewrote to explain why the sort post-processor cannot place inter-child tokens back in a stable position after reordering. Signed-off-by: UncleSp1d3r <unclesp1d3r@evilbitlabs.io>
There was a problem hiding this comment.
Warning
CodeRabbit couldn't request changes on this pull request because it doesn't have sufficient GitHub permissions.
Please grant CodeRabbit Pull requests: Read and write permission and re-run the review.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@internal/opnsensegen/template.go`:
- Around line 177-205: The warning messages emitted in the depth==0 switch in
internal/opnsensegen/template.go (the cases handling xml.CharData, xml.Comment,
xml.ProcInst, xml.Directive and using start.Name.Local) are too terse; update
each log.Warn call to include a clear, actionable instruction (e.g. mention the
dropped token type, the affected section from start.Name.Local, and what the
operator should do to preserve the annotation such as "move this comment into
managed section or use override/patch mechanism"), and for xml.ProcInst include
tok.Target as now but append the same actionable guidance so operators know how
to avoid losing annotations.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 44c08d87-5f03-412a-a25c-bc9145d4d352
📒 Files selected for processing (8)
cmd/cmd_test.gocmd/generate.gointernal/faker/device.gointernal/faker/device_test.gointernal/faker/options.gointernal/opnsensegen/commondevice_test.gointernal/opnsensegen/template.gointernal/serializer/opnsense/serializer_test.go
| default: | ||
| if depth > 0 { | ||
| current.tokens = append(current.tokens, xml.CopyToken(t)) | ||
| continue | ||
| } | ||
| // At depth 0 (between children) we silently drop indentation | ||
| // whitespace because the encoder re-indents on emit. A | ||
| // non-whitespace token here (user-authored comment, CDATA, | ||
| // processing instruction) is dropped with a warning so | ||
| // operators can see annotations were lost from their base | ||
| // config — the sort post-processor cannot place inter-child | ||
| // tokens back in a stable position after reordering. | ||
| switch tok := t.(type) { | ||
| case xml.CharData: | ||
| if len(bytes.TrimSpace(tok)) > 0 { | ||
| log.Warn("dropping non-whitespace chardata in map-backed section", | ||
| "section", start.Name.Local) | ||
| } | ||
| case xml.Comment: | ||
| log.Warn("dropping XML comment in map-backed section (sort post-processor does not preserve comments)", | ||
| "section", start.Name.Local) | ||
| case xml.ProcInst: | ||
| log.Warn("dropping XML processing instruction in map-backed section", | ||
| "section", start.Name.Local, "target", tok.Target) | ||
| case xml.Directive: | ||
| log.Warn("dropping XML directive in map-backed section", | ||
| "section", start.Name.Local) | ||
| } | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Silently dropping comments may surprise users overlaying customized base configs.
When operators add XML comments inside <interfaces> or <dhcpd> in their base config for documentation purposes, those comments are dropped with only a warning log. This is documented behavior, but the warning message could be more actionable.
♻️ Optional: More actionable warning message
case xml.Comment:
- log.Warn("dropping XML comment in map-backed section (sort post-processor does not preserve comments)",
- "section", start.Name.Local)
+ log.Warn("dropping XML comment in map-backed section; move comments outside <interfaces>/<dhcpd> to preserve them",
+ "section", start.Name.Local)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@internal/opnsensegen/template.go` around lines 177 - 205, The warning
messages emitted in the depth==0 switch in internal/opnsensegen/template.go (the
cases handling xml.CharData, xml.Comment, xml.ProcInst, xml.Directive and using
start.Name.Local) are too terse; update each log.Warn call to include a clear,
actionable instruction (e.g. mention the dropped token type, the affected
section from start.Name.Local, and what the operator should do to preserve the
annotation such as "move this comment into managed section or use override/patch
mechanism"), and for xml.ProcInst include tok.Target as now but append the same
actionable guidance so operators know how to avoid losing annotations.
Summary
opnConfigGenerator now implements the reverse serializer that opnDossier doesn't ship. opnDossier parses
config.xml → *model.CommonDevice; this PR delivers the missing inverse direction:faker → *model.CommonDevice → *opnsense.OpnSenseDocument → config.xml. Zero-argumentopnConfigGenerator generateemits a valid OPNsenseconfig.xmlthat round-trips through opnDossier'sConvertDocumentwith zero warnings.Impact: 57 files changed, +2,513 / −3,615 lines (net −1,102). One new dependency (
gofakeit/v7). Two new packages, one rewritten package, two deleted packages.Risk level: 🟡 Medium — architectural pivot with breaking CLI changes; mitigated by round-trip acceptance test, byte-stability tests, fuzz tests, and cross-platform CI green.
Review time: ~60–90 min (11 commits; recommend reading commit-by-commit).
Type of Change
CommonDevice → OpnSenseserializer is a net-new deliverableopnsensegendocs/solutions/entryWhat Changed
🔧 New packages
internal/faker/(7 files + tests) — produces*model.CommonDeviceviaNewCommonDevice(opts...). Functional options:WithSeed,WithVLANCount,WithFirewallRules,WithHostname,WithDomain,WithDeviceType. Target-neutral by default. Usesgofakeit/v7+math/rand/v2PCG for seeded determinism.internal/serializer/opnsense/(7 files + tests) —Serialize(device) → *opnsense.OpnSenseDocumentandOverlay(base, device) → *opnsense.OpnSenseDocumentfor the--base-configpath. One file per CommonDevice subsystem; package layout reserves aninternal/serializer/pfsense/sibling for a future plan.🔧 Rewritten packages
internal/opnsensegen/template.go— transport-only (LoadBaseConfig,ParseConfig,MarshalConfig).MarshalConfigpost-processes XML to sort children of map-backed sections (<interfaces>,<dhcpd>) alphabetically so output is byte-stable under a fixed seed. Atomic single-write contract: encode errors leave the destination untouched.internal/csvio/csvio.go— consumes*model.CommonDeviceinstead of[]generator.VlanConfig. German headers preserved.cmd/generate.go— zero-argument emission, new flag surface, overlay routing helper.🗑️ Deleted packages
internal/generator/(14 files) — dead code under the new pipeline.internal/validate/(2 files) — only validatedgenerator.VlanConfig.📝 Documentation
README.mdreframes the project as the opnDossier reverse serializer.CONTRIBUTING.mdadds an "Adding a New CommonDevice Subsystem" playbook.GOTCHAS.md§7.1 documents the map-backed-section ordering workaround; §7.2 documents the round-trip-must-assert-per-field lesson.Breaking Changes
Removed CLI flags
--count,--csv-file,--firewall-nr,--opt-counter,--include-firewall-rules,--firewall-rule-complexity,--vlan-range,--vpn-count,--nat-mappings,--wan-assignments.New CLI surface
--formatxml(was required)xmlorcsv--vlan-count/-n10--base-config--firewall-rulesfalse--seed0(random)--hostname,--domainTesting
Round-trip contract
TestRoundTripininternal/serializer/opnsense/serializer_test.gois the primary acceptance gate. It exercises the full pipeline —faker → Serialize → MarshalConfig → ParseConfig → ConvertDocument— and asserts per-field parity (not just counts) on:opnDossier returns zero
ConversionWarnings on every round-trip.Byte stability
TestRoundTripByteStablere-marshals the same input 20 times and asserts byte-identical output — a direct guard forsortMapBackedSectionsagainst Go's randomized map iteration.Determinism
TestGenerateDeterministicSeedcompares two--seed 42runs byte-for-byte.TestNewRandZeroSeedIsRandomsamples 8 draws to drive flake probability to ~2⁻⁵¹².Fuzz + boundaries
TestNewCommonDeviceFuzzSeedsiterates 200 distinct seeds at--vlan-count 8to catch adversarial-seed regressions. CLI boundary tests cover--vlan-count 0,1,4094(reject), malformed--base-configXML, and overlay wholesale-replace.Cross-platform
CI green on
ubuntu-latest,macos-latest,windows-latest, plus CodeQL and DCO checks.Review Checklist
Architecture
*model.CommonDeviceis the single intermediate representation; no parallel typesWithDeviceTypeoption, not hardcoded)internal/serializer/opnsense/) reserves pfSense siblinginternal/opnsensegen/) has no generation or serialization logicCorrectness
sortMapBackedSectionscovers every map-backed section inOpnSenseDocument(currentlyinterfaces,dhcpd)MarshalConfigbuffers fully before firstWrite— no partial output on encode failurepickUnique*loops return errors on exhaustion, not panicCLI
generateemits valid XML to stdout--base-configonly accepted with--format xml--seed Nproduces byte-identical output across invocationsTests
require.Trueon map key presence before reading fieldsDocs
Risk Assessment
gofakeit/v7), no CGOMitigations:
GOTCHAS §7.2).Pipeline Architecture
flowchart LR CLI[cmd/generate] --> F[faker.NewCommonDevice] F --> CD[*model.CommonDevice] CD --> S[serializer/opnsense.Serialize] CD -- "--format csv" --> CSV[csvio.WriteVlanCSV] S --> D[*opnsense.OpnSenseDocument] BASE[--base-config file] --> LOAD[opnsensegen.LoadBaseConfig] LOAD --> OV[serializer/opnsense.Overlay] CD --> OV OV --> D D --> M[opnsensegen.MarshalConfig] M --> XML[(config.xml)] CSV --> OUT[(csv)]Known Residuals (Pending Todos)
Captured in
.context/compound-engineering/todos/(gitignored, local-only) for follow-up triage. None block merge — they were surfaced by ce:review asgated_autoormanualand need a product call before implementation:cmd/root.goserializer/opnsense/overlay.goSystem.User/Group(merge vs warn+document decision)serializer/opnsense/overlay.gocmd/validate.govalidatesubcommand still stub; wire to opnDossier or hidecmd/generate.gointernal/faker/rand.go--seed 0sentinel ambiguityinternal/csvio/csvio.gocsv.Readercmd/root.go--output -stdout conventionEach will be addressed in a focused follow-up PR.
Deferred Subsystems (Follow-up Plans)
Phase 1 covers System, Interfaces, VLANs, DHCP, and default firewall rules. One plan per subsystem, landing as separate PRs:
NAT · VPN (OpenVPN / WireGuard / IPsec) · Users/Groups · Certificates/CAs · IDS · HighAvailability · VirtualIPs · Bridges · GIF/GRE/LAGG · PPP · CaptivePortal · Kea DHCP · Monit · Netflow · TrafficShaper · Syslog forwarding · pfSense target.
Test Plan (reviewer verification)
just ci-checkpasses locallygo run . generate | heademits valid XMLgo run . generate --seed 42twice produces identical bytesgo run . generate --base-config testdata/base-config.xmlpreserves base fields outside Phase 1 scopeCommits (chronological)
53131f3d5d8cae0460e3dcd3b179dd0f27cde4ffbab10092454787a63756d966b660fb2f615de